Firstly, a multimodal recognition system utilizing the camera, microphone array, etc. was developed. Next, in order to set a technological foundation to facilitate the interaction of the robot with the human, a conversation control system was developed that can control the speech, motion, gaze, and emotion of the robot based on its intention and desire towards making the human feel more human-like existence of the robot during the interaction. Although the experiment for the verification of the system was conducted for a short period of time, including having conversation with a visitor in a waiting room; it has proved that the android "ERICA" is able to conduct natural conversation and increase the perceived existence of the robot by the human, which are less likely to be achieved by using the other well-known robots. Furthermore, by using some novel technologies such as the implementation of natural and various types of nodding during the interaction, asking in return with analyzing the linguistical focus terms of the interaction sentence, and the implementation of the reaction detection mechanism, a conversation system was developed for the robot which has resulted in more human-like sense of conversing. Adopting this system in an experiment in which human participants were asked to have a conversation with the robot, and the human participants were interviewed by an interviewer during and after the experiment, a successful induction of the human was approved in speaking with the robot and continuing a human-like conversation for a long period of time; compared to the well-known smart speaker-based systems.

The new research stems from a symbiotic human-robot interaction project from the Japan Science and Technology Agency. This is based around the development of a multimodal conversation control system together with a multi-robot conversation control system. These have been designed to create a robot that possess a much higher degree of human-like presence than any comparable robot today. A secondary aim is to create a robot with a'sense of conversing'. This outcome of the project is to design a new generation of conversational robots, starting with a child-like android dubbed'ibuki'.